Transistors are basic building elements in integrated circuits. In previous development of the integrated circuits, the gates of transistors are migrating from polysilicon gates to metal gates, which are typically formed as replacement gates. The formation of the replacement gates involves forming dummy gate stacks, forming gate spacers on sidewalls of the dummy gate stacks, removing the dummy gate stacks to form openings between the gate spacers, depositing gate dielectric layers and metal layers into the openings, and then performing a Chemical Mechanical Polish (CMP) process to remove excess portions of the gate dielectric layers and the metal layers. The remaining portions of the gate dielectric layers and the metal layers are replacement gates.